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1.  INTRODUCTION 

One  of  the  most  significant  air  traffic 
challenges  is  managing  the  National  Airspace 
System  (NAS)  in  a  manner  that  optimizes 
efficiency  and  mitigates  avoidable  delay,  while 
maintaining  safety,  when  convective  weather  is 
present  To  do  this,  aviation  planners  seek  to 
develop  strategic  air  traffic  management  (ATM) 
plans  and  initiatives  that  anticipate  weather 
constraints  2-8  hours  in  the  future  and  identify 
options  and  alternatives  for  efficient  operations 
during  the  off-nominal  NAS  conditions.  With 
accurate  predictions  of  weather  constraints, 
aviation  planners  can  execute  efficient  strategic 
traffic  management  initiatives  such  as  Airspace 
Flow  Programs  (AFP;  Brennan,  2007),  Ground 
Delay  Programs,  Strategic  “Playbook”  reroutes, 
etc.,  in  an  effort  to  mitigate  avoidable  air  traffic 
delay. 

In  support  of  strategic  planning,  traffic 
managers  currently  conduct  bi-hourly  Strategic 
Planning  Telecoms  (SPTs)  and  devise  weather 
impact  mitigation  plans  using  the  human¬ 
generated  Collaborative  Convective  Forecast 
Product  (CCFP).  However,  most  operational 
decision-makers  agree  that  the  quasi-deterministic 
CCFP  "polygons”  (accompanying  by  a  "low/high” 
forecast  confidence  rating)  lack  the  granularity  and 
temporal  resolution  to  adequately  support  efficient 
strategic  ATM  plans  and  decisions. 

Recent  advances  in  strategic  weather  forecast 
technology  include  the  CoSPA  forecast.  CoSPA  is 
a  high  resolution  (3  km)  0  -  8  hr  deterministic 
forecast  (updating  every  15  minutes)  of 
precipitation  and  echo  tops.  CoSPA  uses  Corridor 
integrated  Weather  System  (CIWS)  short-term 
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forecast  techniques,  numerical  weather 
predictions  from  the  High  Resolution  Rapid 
Refresh  (HRRR)  model  for  longer-term  forecasts, 
and  technologies  to  "blend'1  the  two  for  its 
seamless  radar-like  presentation  of  storm 
evolution.  A  detailed  description  of  the  CoSPA 
forecast  technology  and  displays  is  presented  in 
Section  2  of  this  paper. 

In  2010,  a  field  evaluation  was  conducted  at 
17  FAA  and  airline  dispatch  facilities  to  assess  the 
ATM  strategic  planning  utility  and  estimated  delay 
reduction  benefits  of  the  CoSPA  weather  forecast. 
Simultaneous,  real-time  observations  of  CoSPA 
usage  were  made  on  15  convective  weather  days 
during  June  -  September  2010,  A  description  of 
the  design  and  methodology  of  this  experiment, 
including  details  on  how  cost  savings  were 
estimated  for  observed  beneficial  decisions 
derived  from  CoSPA,  is  presented  in  Section  3  of 
this  paper. 

The  observed  CoSPA  operational  benefits 
included  more  efficient  strategic  planning 
initiatives  (such  as  AFP  and  Playbook  reroute 
execution),  enhanced  common  situational 
awareness  of  anticipated  weather  constraints  on 
airspace  capacity,  and  improved  collaborative 
decision-making  when  developing  strategic  plans. 
CoSPA  was  also  used  frequently  by  Center 
Weather  Service  Unit  (CWSU)  meteorologists  in 
FAA  En  Route  Centers  and  weather  specialists  at 
the  Air  Traffic  Control  System  Command  Center 
(ATCSCC)  to  assist  with  their  tasks  -  including 
CCFP  collaboration  and  development.  Results 
demonstrating  the  frequency  of  CoSPA  use  at 
each  facility,  observed  delay  mitigation  and 
decision  coordination  benefits  based  on  use  of  the 
forecast,  and  quantified  delay  savings  (per  use 
and  as  an  annual  estimate)  are  presented  in 
Section  4  of  this  paper. 

2.  COSPA  FORECAST  TOOL 

CoSPA  is  a  high  resolution  (3  km)  0-8  hour 
forecast  (updated  every  15  minutes)  of  Vertically 
Integrated  Liquid  Water  (VIL)  precipitation  and 


echo  tops.  CoSPA  capabilities  available  for  the 
2010  Operational  Evaluation  included: 

1)  Current  and  Past  Weather:  Current 
weather  and  up  to  eight  hours  of  past 
weather  including  satellite,  VIL 
precipitation  and  echo  tops  for  the 
Contiguous  United  States  (CONUS). 
Resolution  is  1  km  and  update  rate  is  2.5 
min.  Additional  weather  information 
included: 

a.  Cloud-to-ground  Lightning  Data  (1  min 
update  rate) 

b.  Storm  Motion  Vectors,  Echo  Top 
Tags,  and  Storm  Growth  and  Decay 
Trends  Contours  (2.5  min  update  rate) 

c.  Standard  and  Winter  Precipitation 
settings 

2)  Corridor  Integrated  Weather  System 
(CIWS)  Forecasts:  0-2  hour  (5  minute 
increments)  deterministic  forecast  of  VIL 
precipitation  and  echo  tops  for  the 
CONUS.  Additional  features  included: 

a.  Forecast  and  verification  contours  for 
precipitation  and  echo  tops 

b.  5  minute  update  rate  at  1  km 
resolution 

c.  Forecast  Accuracy  Scores  for 
precipitation  and  echo  tops 

d.  Both  Standard  and  Winter 
Precipitation  settings 


3)  CoSPA  Forecast:  2-8  hour  deterministic 
forecasts  of  precipitation  and  echo  tops  for 
the  CONUS.  Forecast  information 
included: 

a.  Forecast  and  verification  contours  for 
precipitation  and  echo  tops 

b.  15  minute  update  rate  at  3  km 
resolution 

4)  CCFP  Overlay:  The  2,  4,  and  6  hour 
CCFP  polygons  were  provided  as  an 
overlay  on  the  CoSPA  forecast  product. 

The  CIWS/CoSPA  situation  display  (SD)  was 
used  to  show  the  current  weather  situation  and 
forecasts  of  precipitation  and  echo  tops  via 
forecast  animation  loops,  static  forecast  images, 
and  contour  overlays.  The  SD  functionality  was 
robust,  with  agile  panning  and  zooming  available 
via  mouse  control.  An  operator  could  focus  the  SD 
to  the  terminal  level  with  pixel  resolution  of  1  km 
(0.5  nm)  and  continuously  zoom  out  to  full  CONUS 
with  7  km  resolution  (4  nm)  pixels. 

Users  had  the  option  of  viewing  the  forecast  in 
both  tactical  and  strategic  modes  (See  Figure  2-1). 
The  tactical  mode  (“2hr  Fcst”  button)  showed 
CIWS  0-2  hr  forecasts  for  the  full  CONUS  and  was 
primarily  intended  for  shorter  term  operational 
decisions.  In  the  tactical  mode,  two  hours  of  past 
weather  was  available  in  an  animation  loop  and 
the  forecast  frame  rate  was  adjustable  from  5 
minutes  to  1  hour. 


Figure  2-1.  CoSPA  and  its  associated  features  (shown  here  with  8-hr  precipitation  forecast  enabled). 


The  CoSPA  strategic  mode  (“8hr  Fcst”  button) 
is  intended  for  longer-term  planning  as  well  as 
short-term  situational  awareness.  For  the  strategic 
mode,  the  display  could  run  in  an  animation  loop 
mode,  with  up  to  8  hours  of  past  weather  that 
transitioned  into  a  forecast  with  8  hours  of  future 
weather  for  the  full  CONUS.  The  user  could  also 
adjust  the  frame  rate  frequency  to  15  min,  30  min, 
or  60  min  time  steps.  To  assess  the  performance 
of  CoSPA  forecasts,  previous  forecasts  could  be 
evaluated  by  activating  the  “Verification”  contours 
product  on  the  CoSPA  SD.  With  “Verification” 
enabled,  forecasted  contours  of  level  3+ 
precipitation  or  30  kft+  echo  tops,  valid  at  the 
current  time,  are  overlaid  on  the  current 
precipitation  and  echo  tops  images,  respectively, 
for  direct  comparison. 

Additional  CoSPA  weather  features  that  could 
be  enabled  on  past  and  current  weather  frames  of 
the  CoSPA  animation  include  key  products  such 
as  Growth  and  Decay  Trends  and  Echo  Tops 
Tags.  The  “G&D  Trends”  product  shows  current 
storm  growth  and  decay  areas  that  had  been 
changing  over  the  past  15  to  18  min.  The  Echo 
Tops  Tags  allows  users  to  view  the  estimated 
storm  top  heights,  with  Ikft  resolution,  as  an 
overlay  on  the  Precip  or  Echo  Tops  gridded 
products. 

The  CoSPA  forecast  (Wolfson  et  al.  2007)  is 
comprised  of  three  primary  components:  the 
heuristic  extrapolation  forecast,  the  HRRR 
numerical  model,  and  the  blending  algorithm 
(Dupree  et  al.  2009;  Weygandt  et  al.  2010;  Pinto 
et  al.  2010).  For  the  0-2  hour  forecast,  CIWS 
technology  is  used. 

Extrapolation  Forecast:  The  CoSPA 

extrapolation  technique  was  designed  to  adapt 
and  improve  upon  the  advection  techniques 
used  in  CIWS.  The  0-2  hour  forecast  uses  an 
Eulerian-like  extrapolation  technique  which 
works  well  for  short-term  motion;  however,  to 
obtain  forecasts  out  to  8  hours,  larger  scales 
are  necessary.  The  motion  prediction  consists 
of  three  fundamental  steps:  1)  filtering  and 
tracking,  2)  interpolation  of  motion  fields,  and 
3)  advection  of  the  weather  (see  Dupree  et  al. 
2009). 

High  Resolution  Rapid  Refresh  (HRRR):  An 
experimental  version  of  the  Weather  Research 
and  Forecasting  (WRF)  model  called  the 
HRRR  model  is  being  run  at  the  National 
Oceanic  and  Atmospheric  Administration 
(NOAA)  Earth  System  Research  Laboratory 
(ESRL)  Global  Systems  Division  (GSD) 


laboratory.  The  HRRR  model  is  a  3-km 
resolution  model  that  is  nested  inside  an 
experimental  version  of  the  13-km  Rapid 
Update  Cycle  (RUC)  model.  This  version  of 
RUC  assimilates  three  dimensional  radar 
reflectivity  data  with  a  method  based  on  a 
diabatic  Digital  Filter  Initialization  (DFI) 
technique  (Weygandt  et  al.  2010).  The  HRRR 
model  is  restarted  every  hour  and  generates 
fresh  forecasts  out  to  15  hours.  HRRR  VIL 
forecasts  have  been  made  available  at  a 
special  sub-hourly  (15  minute)  frequency  for 
the  CoSPA  forecast  system  to  best  take 
advantage  of  the  blending  technology.1 

Blending:  The  blending  algorithm  has  been 
designed  to  combine  extrapolation  and  model 
forecasts  of  VIL  to  produce  a  seamless, 
rapidly  updating  0-8  hour  forecast  of  weather 
intensity  (Phillips  et  al.  2010).  The  blending 
process  optimizes  the  overall  skill  of  the 
forecast  by  taking  advantage  of  the  strengths 
of  both  the  extrapolated  and  numerical  model 
forecasts.  The  extrapolated  forecast  tends  to 
be  more  accurate  in  shorter  time  horizons  (0-4 
hours)  and  the  numerical  model  tends  to  be 
more  accurate  in  the  longer  time  horizons  (3-8 
hours).  To  provide  an  optimized  forecast, 
blending  is  done  through  a  calibration  of 
model  data,  a  phase  correction  to  remove 
location  errors  in  the  model  and  statistically- 
based  weighted  averaging.  In  CoSPA, 
heuristic  extrapolation  forecasts  of  VIL  are 
blended  with  VIL  forecasts  from  the  HRRR 
model. 

3.  COSPA  FIELD  EVALUATION  AND 
BENEFITS  ASSESSMENT  METHODOLOGY 

The  CoSPA  operational  benefits  study  was 
modeled  after  the  delay  reduction  methodology 
used  for  CIWS  and  the  New  York  Route 
Availability  Planning  Tool  (RAPT)  in  2003  -  2009 
(Robinson  et  al.  2004;  Robinson  et  al.  2006; 
Robinson  et  al.  2008;  Robinson  et  al.  2009). 
Knowledgeable  observers,  with  in-depth 
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blended  with  the  HRRR  forecasts  closest  to  the  valid 
time  of  the  extrapolation  forecasts. 


understanding  of  how  to  use  and  interpret  CoSPA 
and  with  experience  in  assessing  the  weather- 
ATM  decision-making  environment,  were  present 
at  several  FAA  and  airline  facilities  during 
convective  weather  events  in  order  to  observe  the 
operational  usage  and  performance  of  both 
strategic  weather  forecasts.  Observations  at  each 
facility  were  made  simultaneously  in  order  to 
better  understand  the  coordination  and 
collaboration  interactions  associated  with  strategic 
weather  impact  mitigation  planning. 

3.1  Participating  Facilities  and  In-Field 

Assessment  Events 

In  an  effort  to  conserve  resources  while  still 
assessing  the  forecast  capabilities  across  a  larger 
NAS  region  (with  varying  airspace  management 
concerns  and  weather  impact  characteristics), 
observations  of  CoSPA  use  in  the  field  were 
conducted  using  a  regional  approach.  Field 
observation  days  were  targeted  as  EAST  or 
MIDWEST  evaluation  events,  with  a  different 
group  of  facilities  included  for  forecast  field-use 
assessments  depending  on  the  regional  focus. 
Table  3-1  lists  the  FAA  and  airline  facilities 
included  for  EAST  and  MIDWEST  evaluation 
events,  along  with  the  field  evaluation  dates  for 
both  regions  in  2010.  Collectively,  observation 
teams  from  the  FAA  Aviation  Weather  Office,  MIT 
Lincoln  Laboratory,  the  National  Center  for 
Atmospheric  Research  (NCAR),  and  AvMet 
Applications  were  dispatched  to  17  traffic 
management  facilities  that  included  the  FAA 
Command  Center  (ATCSCC)  and  a  mix  of  En 
Route  Centers  (ARTCCs),  large  terminals 
(TRACONs),  and  commercial  air  carriers 
considered  primary  decision-makers  for  strategic 
ATM  planning. 

Observations  of  CoSPA  use  in  the  field  were 
conducted  on  15  convective  weather  days  (181 
hours  of  air  traffic  control  operations)  during  the 
2010  summer  storm  season.  Convective  weather 
coverage,  location,  storm  type,  intensity,  and  times 
of  storm  development  and  decay  (overall 
evolution)  varied  substantially  across  the  15 
observations  days  (listed  in  Table  3-1),  resulting  in 
a  large  variety  of  air  traffic  impacts  and  ATM 
strategic  planning  initiatives  to  help  mitigate  delay. 
Therefore,  these  observation  periods  were 
considered  a  representative  sample  of  the 
convective  weather  events  that  can  disrupt  air 
traffic  operations  in  the  critical  Midwest  and  East 
Coast  travel  corridors  of  the  NAS. 


During  convective  weather  events,  observers 
at  each  participating  facility  routinely  documented 
the  following  information: 

•  Weather  conditions  in  or  near  the  facility 
or  airspace  of  interest; 

•  Air  traffic  issues  or  problems  as  a  result  of 
weather; 

•  Air  traffic  impact  mitigation  strategies, 
initiatives,  or  programs  in  place; 

•  Weather  information  being  used  to  support 
air  traffic  strategic  decision-making; 

•  Specific  information  provided  by  all 
strategic  weather  forecast  tools  (CoSPA, 
CCFP,  etc.); 

•  Specific  uses  of  strategic  weather  forecast 
products  (CoSPA,  CCFP,  etc.); 

•  Impact  of  CoSPA,  CCFP,  and  use  of  other 
forecasts  on  air  traffic  strategic  decision¬ 
making;  and 

•  Alternative  traffic  management  decisions 
had  the  CoSPA  decision  not  been  made. 

Observations  were  made  several  times  per 
hour  as  the  anticipated  convective  weather  event 
evolved.  Observations  were  most  frequent  during 
the  morning  hours,  when  strategic  plans  were 
being  developed,  and  strategic  weather  forecasts 
were  used  most  often  to  prepare  for  afternoon  and 
evening  traffic  pushes  coinciding  with  increased 
(often  diurnally-driven)  thunderstorm  development. 

The  observers  at  the  FAA  and  airline  facilities 
were  also  available  for  additional,  live  CoSPA 
training.  In  order  to  build  confidence  in  the  forecast 
capabilities  and  to  increase  user  experience,  on- 
the-spot  training  was  provided  when  questions 
arose.  Observers  took  great  care  to  note  all 
instances  when  CoSPA  training  or  additional 
assistance  was  provided.  These  instances  were 
removed  from  the  benefits-observations  database 
in  order  to  ensure  that  CoSPA  benefits 
calculations  presented  in  this  report  were  based 
solely  on  unassisted  usage  of  this  forecast 
capability. 


Table  3-1 

CoSPA  EAST  and  MIDWEST  Field  Evaluation  Sites  and  Dates 


EAST 

MIDWEST 

ATCSCC 

ATCSCC 

Boston  ARTCC  (ZBW) 

Minneapolis  ARTCC  (ZMP) 

Washington  DC  ARTCC  (ZDC) 

Chicago  ARTCC  (ZAU) 

Indianapolis  ARTCC  (ZID) 

Kansas  City  ARTCC  (ZKC) 

NY  ARTCC  (ZNY) 

Chicago  TRACON  (C90) 

Cleveland  ARTCC  (ZOB) 

Continental  Airlines  AOC  (COA) 

Atlanta  ARTCC  (ZTL) 

United  Airlines  AOC  (UAL) 

NYTRACON  (N90) 

Delta  Air  Lines  AOC  (DAL) 

Continental  Airlines  AOC  (COA) 

. 

Delta  Air  Lines  AOC  (DAL) 

Jet  Blue  Airlines  AOC  (JBU) 

TOTAL:  11 

TOTAL:  8 

Eval  - 1 

13-Jun 

Eval  - 1 

06-Jul 

14-Jun 

07-Jul 

Eval  -  2 

16-Jun 

08-Jul 

Eval  -  3 

19-Jul 

Eval  -  2 

03-Aug 

20-Jul 

Eval  -  3 

01-Sep 

21-Jul 

02-Sep 

Eval  -  4 

04-Aug 

ATCSCC:  Air  Traffic  Control  System  Command  Center 
ARTCC:  Air  Route  Traffic  Control  Center 

TRACON:  Terminal  Radar  Control  Facility 

AOC:  Airline  Operations  Center 

05-Aug 

Eval  -  5 

16-Sep 

3.2  METHODOLOGY  FOR  ESTIMATING 

DELAY  REDUCTION  BENEFITS 

Using  the  methodology  summarized  in  Figure  3-3, 
the  detailed  observations  collected  during  the 
EAST  and  MIDWEST  evaluation  periods  were 
used  to  determine  the  various  CoSPA  operational 
benefits,  to  estimate  the  frequency  of  each 
observed  benefits  category,  and  to  estimate  delay 
savings  per  beneficial  decision.  The  most 
significant  component  of  this  methodology  is  the 
case  study  analysis  of  individual  situations  where 
CoSPA  was  the  primary  weather  forecast  tool 
used  to  make  strategic  planning  decisions.  With 


these  case  studies,  it  was  important  to  not  only 
assess  the  decision  made  based  upon  prototype 
forecast  information,  but  to  also  assess  what 
decision  and  outcome  may  have  occurred  had  the 
CoSPA-derived  decision  not  been  made.  The 
difference  in  traffic  management  outcome  and  the 
estimated  differences  in  delay  between  the 
“beneficial”  decision  and  the  alternative  or 
“baseline"  decision  is  the  estimated  delay 
reduction  benefit  for  a  specific  CoSPA  strategic 
planning  application. 


Figure  3-3.  Methodology  used  to  estimate  annual  CoSPA  delay  savings. 


3.3  Estimating  Quantitative  Delay  Savings 

3.3. 1  Delay  Reduction  Models 

Delay  savings  attributed  to  CoSPA  usage  were 
achieved  both  while  flights  were  airborne  and 
while  they  were  still  on  the  ground.  In  assessing 
these  benefits,  delay  savings  were  considered 
using  both  linear  and  queuing  delay  models. 

The  linear  delay  model  corresponds  to  a 
''transient"  event  where  there  is  no  reduction  in  the 
overall  average  rate  of  aircraft  movement 
(Robinson  et  al.  2004).  Examples  of  linear  delay 
benefits  include  (a)  a  group  of  flights  avoiding  a 
longer  airborne  reroute  and  (b)  flights  released 
from  an  Estimated  Departure  Clearance  Time 
(EDCT)  delay  and  departing  without  excessive  taxi 


delay.  As  described  in  Robinson  et  al.  (2004),  the 
key  element  of  these  types  of  delays  (and  delay¬ 
modeling  exercises)  is  that  the  benefit  for 
improved  performance  is  typically  “linear"  in  each 
of  the  pertinent  variables  (e.g.,  traffic  density, 
ability  to  realize  benefit  in  a  given  ATM  situation, 
rate  of  delay  savings  over  time,  etc.). 

In  estimating  CoSPA  linear  delay  savings,  the 
following  metrics  and  assumptions  were  used: 

•  Passenger  aircraft  average  block  speed: 
365  mph  (FAA,  2010) 

•  Average  aircraft  speed  assigned  to  all 
flights  included  in  benefits  calculations 

®  Any  flights  released  early  from  EDCTs 
(because  of  beneficial  strategic  planning 


decisions)  had  only  linear  delay  savings 
(i.e.,  no  terminal  queuing  delay  savings2). 

The  queuing  model  for  delay  reduction 
estimates  is  applicable  in  situations  where  the 
weather  reduces  the  effective  capacity  of  an 
airspace  resource  (e.g.,  airport  terminal,  fix,  or 
jetway)  for  some  finite  time  while  the  air  traffic 
demand  for  that  resource  is  constant  and  exceeds 
the  weather-reduced  capacity.  Previous  CIWS  and 
RAPT  delay  reduction  benefits  studies  have 
demonstrated  that  both  terminal  and  route-based 
queuing  delays  can  occur  when  convective 
weather  impacts  occur  (Robinson  et  al.  2004; 
Robinson  et  al.  2008).  Given  the  circumstances 
associated  with  the  observed  CoSPA  beneficial 
decision  (see  Section  4),  the  queuing  model  for 
this  study  was  used  primarily  for  route-based 
queuing  delay  scenarios.  In  these  instances  and 
when  applicable,  a  single-server  queue  model3 
requiring  only  two  input  fields  (traffic  demand  and 
capacity  as  a  function  of  time)  and  based  on  the 
following  equation  was  used  to  estimate 
CoSPA/LCH  delay  savings: 

X(Delay  to  individual  aircraft)  =  0.5  T2  (D  -  CW)[(CV 
-CW)/(CV-D)]  (Eq.  3-1) 

Where  D  =  demand,  Cw  =  capacity  during  adverse 
weather,  Cv  =  capacity  during  benign  weather,  and 
T  =  effective  event  duration  (Andrews,  1993). 

In  estimating  route-based  queuing  delay 
savings,  the  following  metrics  and  assumptions 
were  used: 

•  Fair-weather  route  capacity  (traffic 
crossing  a  waypoint): 


2  In  an  effort  to  estimate  CoSPA  delay  reduction  benefits 
as  conservatively  as  possible  (with  the  available 
resources),  it  was  assumed  that  on-the-ground  flights 
included  in  the  benefits  case  studies,  typically  departing 
for  the  large  airports  in  the  Northeast  (e.g.,  metro  NY, 
metro  DC,  PHL),  did  not  encounter  queuing  delays  upon 
departure  -  or  queuing  delay  savings  with  shortened  or 
avoided  EDCT  times  (i.e.,  departure  demand  at  these 
individual  airports  did  not  exceed  available  capacity). 
This  assumption  is  assumed  to  be  valid,  given 
historically  low  air  traffic  demand  in  2010,  and,  given  the 
nature  of  most  of  the  on-the-ground  benefits  case 
studies,  the  exclusion  of  most  airports  prone  to  terminal 
queuing  delays  (e.g.,  metro  NY  airports). 

3  The  queuing  delay  model,  developed  by  Evans  (1997), 
is  discussed  in  detail  in  Robinson  et  al.  (2004).  In  2007, 
the  FAA  Simulation  and  Analysis  Team  (AJP-6610) 
independently  tested  and  validated  this  model,  deeming 
it  suitable  for  weather-ATM  benefits  experiments. 


30  aircraft  /  hr4 

e  Adverse-weather  route  capacity  with  30 
Miles-ln-Trail  (MIT)  restriction: 

10  aircraft  /  hr 

•  Adverse-weather  route  capacity  with  40 
MIT  restriction: 

8  aircraft  /  hr 

3.3.2  Converting  Hours  of  Delay  to  Monetary 
Estimates 

CoSPA  delay  reduction  benefits  expressed  in 
hours  of  delay  were  converted  to  airline  Direct 
Operating  Costs  (DOC)  and  Passenger  Value 
Time  (PVT).  The  following  cost  conversions  and/or 
multipliers  were  used  to  estimate  the  total  2010 
delay  reduction  benefits  attributed  to  CoSPA  or 
LCH: 

1 .  2010  DOC,  on-the-ground,  at  the  gate5: 
$967  per  hr  (FAA,  2010) 

2.  2010  DOC,  airborne: 

$4289  per  hr  (FAA,  2010) 

3.  Commercial,  personal  PVT: 

$23.30  per  passenger  per  hr  (FAA,  2010) 

4.  2010  avg  passengers  per  aircraft  (FAA, 
2010):  54  passengers  per  aircraft 

[(air  carrier  capacity  *  avg  load  factor) 
+  (air  taxi  capacity  *  avg  load  factor)]/2 
-  [(101.2  *  .813)  +(33  *  757)]/2 

5.  2010  PVT: 

$1258  per  hr 

6.  Downstream  delay  multiplier: 


4  Robinson  et  al.  (2004)  assumed  10  Miles-ln-Trail  (MIT) 
as  a  conservative  estimate  of  aircraft  spacing  on  a  jet 
route  outside  of  terminal  airspace  during  fair  weather. 
Again  to  be  conservative,  they  also  assumed  a  fast 
average  ground  speed  (530  mph)  for  flights  included  in 
calculations  of  route  capacity  for  route-based  queuing 
delay  estimates.  Together,  these  values  for  route  usage 
variables  result  in  the  conservative  estimate  of  30 
aircraft  per  hour  on  a  route.  These  same  input  variables 
were  used  in  the  route-based  queuing  delay  estimates 
in  this  report. 

5  The  2010  on-the-ground  airline  DOC  for  passenger  air 
carriers  was  $2054/hr.  However,  this  DOC  accounts  for 
costs  incurred  while  taxing  on  the  airport  surface.  Since 
most  on-the-ground  delay  savings  attributed  to  CoSPA 
were  the  result  of  improved  or  avoided  EDCTs,  the  at- 
the-gate  DOC  metric  was  recommended  as  more 
appropriate  (FAA  ATO-F,  personal  communication). 


1.57  (Welman  etal.  2010) 

7.  Downstream  operating  cost  savings: 

0 

8.  Downstream  PVT  savings: 

0.57  of  direct  PVT 

Calculated  downstream  delay  benefits  were 
also  calculated  to  estimate  the  ripple  effect  that 
arises  when  an  aircraft  is  delayed  on  one  leg  of  a 
flight  (e.g.,  due  to  adverse  weather)  such  that  the 
subsequent  legs  flown  by  that  aircraft  that  day  are 
also  delayed  (e.g.,  DeArmon,  1992).  In  this  study, 
downstream  delay  reductions  are  assumed  to 
equal  57%  of  the  initial  delay  (Welman  et  al. 
2010).  Following  the  rationale  described  in 
Robinson  et  al.  (2004),  downstream  cost  savings 
are  estimated  only  for  PVT.  The  downstream 
delay  savings  multiplier  used  in  this  study  is 
considered  conservative  relative  to  past  research 
in  this  area  (e.g.,  Hartman,  1993;  Beatty  et  al. 
1999;  Robinson  et  al.  2004). 

3.4  Determining  Annual  Delay  Savings 

With  the  methodology  described  in  Fig.  3-3, 
median  delay  savings  per  primary  benefit 
category,  determined  from  individual  benefits  case 
study  results,  are  multiplied  by  the  annual 
estimated  frequency  of  occurrence  of  each  benefit 
type  to  determine  annual  CoSPA  delay  reduction 
benefits.  With  this  approach,  the  observed 
frequency  of  use  for  each  benefit  category  at  each 
of  the  individual  facilities  is  normalized  by  the  total 
number  of  observation  days  to  determine  the 
frequency  of  use  per  convective  weather  day  (see 
Section  4.2).  Multiplying  the  normalized  benefits 
frequency  by  an  average  annual  estimate  for 
weather  impact  events  within  the  evaluation  region 
provides  the  annual  usage  estimates  used  to  “roll¬ 
up”  per  event  delay  savings  to  annual  benefits 
results. 

Robinson  et  al.  (2006)  estimated  annual, 
climatologically-adjusted  convective  weather  days 
for  ARTCCs  comprising  the  Great  Lakes  and 
Northeast  NAS  corridors  -  ZAU,  ZID,  ZOB,  ZDC, 
ZNY,  and  ZBW  (see  Appendix  C  of  that  report). 
These  statistics,  along  with  estimates  based  on 
these  results  for  the  three  additional  ARTCCs 
included  in  the  CoSPA/LCH  benefits  evaluation 
(ZMP,  ZKC,  and  ZTL)  are  presented  in  Table  3-2. 
Given  that  strategic  planning  decisions  made  with 
the  aid  of  strategic  weather  forecasts  (including 
CoSPA  in  this  evaluation)  typically  span  across 
multiple  ARTCCs,  and  include  coordination  with 


neighboring  ARTCCs  that  may  be  free  of 
convective  weather  impacts  for  the  impact  day  in 
question,  the  average  number  of  estimated, 
annual  convective  weather  days  for  all  9  ARTCCs 
(95)  was  considered  the  approximate  number  of 
days  per  year  on  which  identified  CoSPA  or  LCH 
benefits  categories  could  be  achieved. 

One  of  the  identified  CoSPA  benefits 
application  (see  Section  5)  that  is  unique  and  the 
exception  -  where  the  estimated  annual  number  of 
days  in  which  the  opportunities  to  consider  the 
decision  is  far  less  than  95  days  -  was  when  the 
forecast  was  used  for  improved  Airspace  Flow 
Program  (AFP)  execution  and  management. 
Historically,  when  considering  the  use  of  this 
strategic  planning  initiative  for  weather  impacts 
during  the  summer  months,  convective  weather 
impacts  would  typically: 

•  Impact  multiple  ARTCC  in  the  Eastern 
NAS 

•  Cover  (or  were  predicted  to  cover)  large 
air  traffic  areas 

•  Develop  and  mature  during  mid-late 
afternoon,  when  en  route  traffic 
disruptions  would  be  most  significant 

•  Specifically  impact  en  route  ZOB  and  ZDC 
airspace,  west  and  south  of  the  NY 
TRACON  (N90) 

For  weather  impact  mitigation,  the  primary 
AFPs  that  have  typically  been  used  are  Flow 
Constrained  Area  (FCA)  A05,  A06,  A08,  and 
OBI6  For  2007,  2008,  and  2010  (years  for  which 
historical  AFP  usage  data  were  available),  these 
AFPs  were  used  44  times  per  year,  on  average.  If 
one  assumes  that  an  AFP  was  considered  but  not 
used  on  an  additional  25%  of  the  total  annual 
implementation  days,  then  the  total  numbers  of 
“AFP-consideration”  days  per  year  equals  55. 
This  was  the  metric  used  in  this  analysis  to  scale 
up  the  daily,  normalized  CoSPA  AFP  benefits 
frequency  of  use  to  the  estimated  annual 
frequency  of  occurrence. 


Graphical  depictions  of  the  pre-determined  AFPs  can 
be  found  at: 

http://www.flv.faa.Qov/What  s  New/AFPs.ppt 


Table  3-2 

Estimated  Annual  ARTCC  Convective  Days, 
Accounting  for  Climatology  * 


ARTCC 

Annual  Convective 
Weather  Days 

ZAU 

110 

ZID 

100 

ZOB 

90 

ZDC 

100 

ZNY 

65 

ZBW 

65 

ZMP• ** 

100 

ZKC  ** 

110 

ZTL  ** 

120 

Average 

95 

*Statistics  from  Robinson  et  al.  (2006),  rounded 
down  to  nearest  multiple  of  5 
**ZMP,  ZKC,  and  ZTL  estimated  based  on  annual 
convective  weather  days  at  nearby 

ARTCCs  (e.g.,  ZAU,  ZDC)  and  known 
tendencies  for  more  (ZKC,  ZTL)  or  less  (ZMP) 
convection  compared  to  the  neighboring 
airspace. 

4.  COSPA  BENEFITS  EVALUATION 

RESULTS 

CoSPA  operational  benefits  were  identified 
directly  from  real-time  observations  of  the  usage  of 
these  weather  forecasts  and  traffic  management 
decision-making  at  all  visited  FAA  and  airline 
facilities  during  the  2010  summer  storm  season. 
From  the  observational  data: 

•  CoSPA  benefits  categories  were  identified 

•  CoSPA  frequency  of  beneficial  decisions 
was  identified 

•  CoSPA  delay  savings  case  studies  were 
conducted 

•  CoSPA  annual  (2010)  delay  savings 
estimates  were  made 

4.1  COSPA  Benefits  Categories 

Usage  of  CoSPA  by  FAA  traffic  managers  and 
airline  dispatch  coordinators  was  partitioned  into 
23  different  benefits  categories,  shown  in  Tables 
4-1  and  4-2.  Benefits  categories  listed  in  Table  4-1 


are  strategic  planning  actions  derived  from  CoSPA 
forecasts  that  can  directly  result  in  delay  savings. 
They  are  therefore  considered  quantifiable 
benefits.  Benefits  categories  listed  in  Table  4-2  are 
impact  awareness  and  planning  support 
applications  derived  from  CoSPA  usage,  and  are 
considered,  at  this  time,  to  be  more  qualitative 
benefits.  It  is  important  to  note  that  an  observed 
CoSPA  application  was  not  assigned  a  benefits 
category  if  the  forecast  was  used  to  assist  with 
decisions  in  the  0  -  2  hour  forecast  period  (i.e., 
“tactical”  planning  period).  Only  unassisted 
strategic  planning  applications,  using  the  2  -  8 
hour  CoSPA  forecast,  were  considered  CoSPA 
benefits  in  this  study.7 

The  three  primary  CoSPA  benefits  categories, 
in  terms  of  observed  frequency  of  beneficial 
decisions  (see  Section  4.2)  and  the  potential  for 
delay  reduction,  were  AFP,  EPB,  and  ERP 
(improved  Airspace  Flow  Program  management, 
enhanced  Playbook  reroute  management,  and 
enhanced  strategic  reroute  planning).  These  three 
categories  were  the  focus  of  the  2010  CoSPA 
delay  savings  estimates.  CoSPA-derived  delay¬ 
saving  decisions  involving  other  quantifiable 
benefits  categories  were  also  evident,  but  in  an 
effort  to  remain  conservative  when  estimating 
CoSPA  benefits,  case  study  results  for  these  other 
categories  were  not  included  in  the  final  delay 
savings  estimates. 

Many  of  the  other  CoSPA  benefits  categories 
listed  in  Table  4-1  pertained  to  strategic  routing 
decisions  for  individual  flights,  city-pairs,  and 
airlines  (e.g.,  ITR,  RIF,  MIL,  and  AFM).  Moreover, 
many  of  these  CoSPA-derived  beneficial  decisions 
were  observed  being  made  by  ATC  coordinators 
and  dispatchers  at  Airline  Operations  Centers 
(AOCs),  such  as  Continental  and  United. 

Though  more  difficult  to  translate  into  tangible 
delay  and  cost  savings  estimates,  the  CoSPA 
benefits  categories  listed  in  Table  4-2  are 
considered  critical  to  improved  strategic  planning 
and  enhanced  collaborative  decision-making 
during  convective  weather  events.  CoSPA 
contributions  to  improved  awareness, 
coordination,  and  collaboration  can  increase  the 
likelihood  of  implementing  other  CoSPA-derived 
capacity  enhancement  decisions  such  as  ERP  or 
AFP. 


7  The  reason  for  this  is  that  the  tactical  ATM  benefits  of 
the  CIWS  0-2  hour  Precipitation  and  Echo  Tops 
forecasts,  which  are  scheduled  to  be  operationally 
available  to  all  traffic  managers  via  the  Traffic  Flow 
Management  System  in  2011,  have  already  been 
evaluated  (e.g.,  Robinson  et  al.  2004). 


Table  4-1 

Observed  CoSPA  Benefits  Categories  (Quantifiable) 


KEY 

CoSPA  BENEFIT  CATEGORY 

*AFP 

Improved  AFP  Execution  /  Management  (assigned  when  CoSPA  used  to  make  AFP 
Go/No-Go  decisions,  AFP  decisions  on  start  time,  stop  time,  rate,  plan  modifications, 
etc.) 

GDP 

Improved  Ground  Delay  Program  Execution  /  Management  (only  assigned  when 
decision  made  to  explicitly  avoid  GDP,  to  implement  GDP,  to  modify  rate/scope,  or  to 
end  GDP,  based  on  CoSPA) 

GS 

Improved  Ground  Stop  Execution  /  Management  (only  assigned  when  decision 
made  to  explicitly  avoid  GS,  implement  a  GS,  modify  scope  or  end  time  of  GS  -  based 
on  CoSPA) 

*EPB 

Enhanced  Playbook  Reroute  Planning  and  Execution  (explicit  action  to  execute, 
modify,  or  avoid  a  Playbook  routing  based  on  CoSPA) 

*ERP 

Enhanced  Reroute  Planning  (includes  avoiding  reroutes  by  recognizing  viability  of 
nominal  routes,  proactive  reroute  implementation,  and  ending  reroutes/returning  to 
nominal  routes  sooner,  etc.) 

MRR 

Improved  Management  of  Route  Restrictions  (includes  MIT  restriction, 
management  on  nominal  routes) 

MIL 

Improved  Strategic  Management  Decisions  in  Support  of  Military  Air  Traffic 

ITR 

International  Traffic  Routing  Assistance  (explicit  routing  decisions  for  international 
traffic  flows  or  individual  flights  to/from  international  destinations) 

RIF 

Strategic  Routing  Assistance  for  Individual  Flights  or  city-pairs 

AFM 

Airline  Fleet  Management  Assistance  (includes  decisions  on  airline-only  ground 
stops,  cancellation  (or  cancellation  avoidance)  decisions,  support  in  issuing  airline 
Planning  Alert  Messages  (PAM),  etc.) 

ICR 

Improved  Integrated  Collaborative  Reroute  (ICR)  Program  execution  and 
management 

DOP 

Departure  Operations  Planning  Assistance  (Large  TRACONs) 

RCM 

Proactive  Runway  Configuration  Management 

TMA 

Enhanced  Planning  for  Airport  Arrival  Metering  (TMA) 

*  Primary  quantifiable  benefits  category 


Table  4-2 

Observed  CoSPA  Benefits  Categories  (Impact  Awareness  and  Planning  Support) 


KEY 

CoSPA  BENEFIT  CATEGORY 

SPD 

Assistance  with  Strategic  Plan  Development 

SP-CCFP 

Use  of  CoSPA  in  concert  with  CCFP  for  enhanced  weather  impact  awareness 

CWSU 

Support  for  CWSU,  SCC  Weather  Unit,  or  NWS  operations  in  support  of  ATM 

l/IC 

Enhanced  Inter/Intra-Facility  Coordination 

STAFF 

Assistance  with  facility  staffing  decisions;  Proactive  Monitor  Alert  Threshold  (MAP) 
limit  adjustments  (aid  sector  controllers) 

EQUIP 

Assistance  with  ATC  equipment  management  decisions 

SA-R 

Enhanced  Situational  Awareness  -  Route  (En  Route  Airspace)  Impact  Monitoring 

SA-T 

Enhanced  Situational  Awareness  -  Terminal  Impact  Monitoring 

SA 

General  Situational  Awareness 

In  addition  to  direct  traffic  management 
support  applications  (e.g.,  SPD,  SA  categories), 
benefits  categories  listed  in  Table  4-2  demonstrate 
the  additional  ATM-support  applications  of  CoSPA 
that  were  observed  during  the  field  evaluations. 
One  of  the  more  significant  findings  was  that 
ARTCC  Center  Weather  Service  Unit  (CWSU) 
meteorologists  were  frequently  observed  using 
CoSPA  precipitation  and  echo  tops  forecasts  to 
assist  with  their  tasks  that  ranged  from  CCFP 
collaboration  and  development,  to  monitoring 
issued  convective  weather  Significant 
Meteorological  information  (SIGMET)  advisories, 
to  direct  input  to  traffic  managers  about  potential 
terminal  or  en  route  weather  impacts  (or 
predictions  for  improving  conditions). 

Another  interesting  observed  CoSPA  "support 
application”  pertained  to  proactive  staffing 
decisions.  Examples  of  these  benefits  included: 

•  ensuring  that  specific  sectors,  predicted  by 
CoSPA  to  be  impacted  by  weather,  would 
have  adequate  (and  sometimes  extra) 
controllers  on  position, 

•  timing  breaks  and  position  rotations  in  an 
ARTCC  Traffic  Management  Unit  (TMU)  to 
support  opening  the  “severe  weather” 
position  when  CoSPA  predicted  weather 
impacts  would  become  significant,  and 

•  proactively  modifying  Monitor  Alert 
Parameter  (MAP)  thresholds,  based  on 
CoSPA  predictions  for  sector  impacts,  to 
ensure  demand  would  be  in  line  with 
expected  capacity  constraints  and 
controllers  working  that  sector  would  not 
be  overloaded. 

4.1.1  Examples  of  Improved  Strategic 

Planning  Benefits  Attributed  to  CoSPA 

During  the  2010  field  evaluation,  operational 
decision-makers  were  observed  repeatedly  using 
CoSPA  to  deliberate  about  the  need  for  and  the 
specific  parameters  of  Airspace  Flow  Programs 
(AFP)  and  strategic  reroutes  [both  with  and 
without  the  use  of  coordinated  Playbook  routings 
(EPB  and  ERP  benefits  categories)].  Strategic 
planners  would  use  the  deterministic  precipitation 
forecast  (on  occasion,  the  CoSPA  echo  tops 
forecast  was  also  consulted)  to  more  specifically 
identify  the  coverage,  location,  and  organization- 
type  [e.g.,  line  (solid  or  broken  -  and  if  broken, 
were  the  gaps  usable  for  traffic  flows),  clusters,  or 
cellular  convection]  of  anticipated  weather  impact 


events.  Operational  users  were  able  to  use  this 
information  to  better  define  the  severity  of  impacts 
within  broader,  less-defined  CCFP  polygons 
(which  were  typically  “low  coverage,  low 
confidence"  predictions). 

Users  also  found  the  rapid  update  rate  of 
CoSPA  to  be  a  positive  forecast  attribute  when 
assessing  the  need  and  scope  for  strategic 
planning  initiatives.  The  availability  of  new,  rapidly- 
updating  CoSPA  forecast  information  proved  to  be 
important  for  strategic  planning,  given  that  traffic 
managers  (ARTCCs)  and  National  Traffic 
Management  Officers  (NTMOs  at  the  ATCSCC) 
were  observed  (during  the  morning  hours)  exerting 
the  most  time  and  effort  assessing  the  potential 
severity  and  possible  solutions  for  anticipated 
weather  impact  events  during  the  30-45  min 
period  prior  to  the  bi-hourly  Strategic  Planning 
Telecoms  (SPT)  -  when  available  CCFP  forecasts 
were  90-105  min  old!8 

Through  these  uses  of  CoSPA,  many 
decisions  were  made  in  support  of  AFP,  Playbook, 
and  strategic  reroute  execution  and  management. 
Specifically,  aviation  planners  were  observed 
using  CoSPA  to  aid  with  decisions  on  the  need, 
timing,  rate,  scope,  duration,  and/or  cancellation  of 
AFP,  EPB,  and  ERP  strategic  planning  initiatives. 
An  example  of  CoSPA  usage  for  AFP  decision¬ 
making  is  illustrated  in  Figure  4-1,  During  this 
event,  ATCSCC  National  Traffic  Management 
Officers  (NTMO)  used  the  CoSPA  forecast  - 
predicting  only  widely  scattered  level  3+ 
precipitation  across  the  Northeast  NAS  (airspace 
being  controlled  by  an  AFP  AOS  program)  over  the 
next  2-6  hours  -  to  make  the  decision  to  cancel 
the  AFP  program  early.  Interestingly,  this  CoSPA- 
derived  decision  was  made  despite  the  CCFP  2-hr 
and  4-hr  forecasts  that  were  available  at  that  time 
predicting  significant  convection  in  the  metro  NY  to 
DC  air  traffic  corridor  (see  Fig.  4-1  A).  By 
cancelling  this  AFP  program  early,  58  flights 
scheduled  to  fly  through  AFP  airspace  had  their 
planned  departure  delays  or  longer  reroutes 


Observations  that  critical  strategic  planning 
assessments  and  collaboration  were  often  conducted 
just  prior  to  the  SPTs,  when  the  “valid"  CCFP  forecasts 
were  most  obsolete,  suggest  that  the  timing  for  when 
CCFP  forecasts  are  developed  and  published  may  need 
to  be  revisited.  Without  up-to-date  CCFP  forecasts 
available  during  their  critical  plan  development  period, 
ATCSCC  NTMOs  were  often  observed  peeking  over  the 
shoulder  of  weather  specialists  in  order  to  read  ongoing 
CCFP  chat  logs  and  to  get  a  preview-glance  of  the 
“next"  CCFP  forecast  -  with  the  CCFP  polygons  still 
under  negotiation. 


cancelled,  allowing  for  considerable  delay  savings 
(Fig.  4-1 B). 

Many  of  the  other  CoSPA  benefits  categories 
listed  in  Table  4-1  pertained  to  strategic  routing 
decisions  for  individual  flights,  city-pairs,  and 
airlines  (e.g.,  ITR,  RIF,  MIL,  and  AFM).  An 
example  of  a  proactive,  strategic  reroute  decision 
made  for  an  individual  flight  using  CoSPA 
forecasts  is  shown  in  Figure  4-2.  In  this  case, 
significant  convection  was  already  impacting  the 
planned  route  of  an  international  flight  about  to 


enter  U.S.  airspace  (Fig.  4-2A).  ZBW  began 
coordinating  with  neighboring  facilities  on  reroute 
options.  At  first,  a  reroute  option  through  the  TN 
valley  was  considered  (white  box  in  Fig.  4-2B),  but 
then  ZBW,  ZNY,  ZDC,  and  ATCSCC  collectively 
used  the  CoSPA  precipitation  forecast  to  identify  a 
viable  reroute  corridor  through  gaps  in  convection 
along  the  East  Coast  (Fig.  4-2B).  The  international 
flight  was  able  to  utilize  this  reroute  without  any 
issues  (see  Figs.  4-2C  and  4-2D). 


Figure  4-1:  (A)  2-hr,  4-hr,  and  6-hr  CoSPA  precipitation  and  CCFP  forecasts  at  2140  UTC  on  16  June 
2010  -  CoSPA/CCFP  forecasts  valid  for  2340/2300  UTC,  0140/0100  UTC,  and  0340/0300  UTC.  (B)  Flight 
Explorer  weather  depiction  and  flight  tracks  at  2305  UTC  on  16  June  2010  showing  flights  (with  tags)  that 
were  released  early  from  Estimated  Departure  Clearance  Time  (EDCT)  delays  and/or  elected  to  not  use 
CAN  7  EAST  reroute,  when  AFP  A05  was  cancelled  early. 


Figure  4-2.  (A)  Flight  Explorer  weather  depiction  and  flight  tracks  at  2100  UTC  on  16  June  2010  showing 
an  international  flight,  AFR434  to  Mexico  City,  entering  ZBW  and  its  filed  route,  using  J174,  blocked  by 
convection.  (B)  CoSPA  2.5-hr  precipitation  forecast  issued  at  2100  UTC  predicts  a  viable  reroute  corridor 
for  AFR434  through  gaps  in  convection  along  the  East  Coast  (blue  line).  Flight  Explorer  flight  tracks  at  (C) 
2230  UTC  and  (D)  2330  UTC  show  that  AFR434  was  able  to  successfully  utilize  this  reroute. 


4.1.2  Insufficient  Situational  Awareness  or 
Strategic  Decisions  Resulting  from  Use  of 
Inaccurate  CoSPA  Forecasts 

Overall,  operational  decisions  made  based  on 
CoSPA  2  -  8  hr  forecasts  resulted  in  more  efficient 
strategic  planning  initiatives  that  helped  reduce  air 
traffic  delay  (see  Section  4.3).  However,  in  some 
instances,  inaccurate  precipitation  or  echo  tops 
predictions  from  this  prototype  forecast  led  to 
impaired  situational  awareness  and  at  times,  poor 
strategic  planning  decisions. 

Figure  4-3  and  4-4  illustrate  an  example  of  a 
poor  planning  decision  made  with  CoSPA.  During 
this  event  (02  Sep  2010),  ZMP  used  the  CoSPA  2 
-  4  hr  precipitation  forecast,  which  showed 
continued  convective  weather  impacts  in 
southeast  ZMP  airspace  (affecting  the  MSP 


southeast  arrival  flow),  to  argue  for  the 
implementation  of  the  MSP_KASPR_EAU 
Playbook  until  1500  UTC  (the  next  4  hours). 
ATCSCC  was  against  this  plan,  but  ZMP  used 
CoSPA  as  evidence  that  the  Playbook  was 
needed  -  and  at  the  time  of  this  decision,  the 
CCFP  forecast  was  almost  2  hours  old  (see  Fig.  4- 
3). 

Unfortunately,  the  CoSPA  forecast  over¬ 
predicted  the  coverage  and  severity  of  convection 
southeast  of  MSP  (Fig.  4-4D).  The  convective 
weather  moved  quickly  northeast,  out  of  this 
region,  and  thus  MSP  arrivals  from  the  southeast 
were  forced  to  fly  unnecessarily  long  routes. 
Moreover,  with  one  MSP  arrival  cornerpost  closed, 
arrival  demand  exceeded  capacity  for  a  time  at 
another  cornerpost,  causing  some  additional 
airborne  holding  (Fig.  4-4B). 


Figure  4-3.  (A)  CoSPA  precipitation  at  1050  UTC  on  02  September  2010,  (B)  +2-hr  CoSPA  precipitation 
(and  +4-hr  CCFP)  forecasts  valid  1250  (1300)  UTC,  and  (C)  +4-hr  CoSPA  precipitation  (and  +6-hr  CCFP) 
forecasts  valid  1450  (1500)  UTC.  Blue  lines  show  the  MSP_KASPR_EAU  Playbook  implemented  for 
MSP  arrival  traffic  (1050  -  1500  UTC)  based  on  the  CoSPA  prediction  for  ongoing  convection  southeast 
of  MSP  airport. 


Figure  4-4.  Flight  Explorer  weather  depiction  and  MSP  arrival  flight  tracks  on  02  September  2010  at  (A) 
1215  UTC,  when  the  MSP  Playbook  was  in  effect,  (B)  1300  UTC,  when  airborne  holding  occurred  as 
demand  for  available  arrival  fixes  exceeded  capacity,  and  (C)  1330  UTC,  when  the  four  arrival  flows 
reopened  early.  (D)  The  3-hr  CoSPA  precipitation  forecast  verification  contour,  compared  with  level  3+ 
"truth"  precipitation,  shows  that  the  large  area  of  convection  predicted  southeast  of  MSP  at  1300  UTC  did 
not  occur. 


Observers  were  not  explicitly  tasked  with 
documenting  CoSPA-derived  decisions  that 
proved  to  be  less  efficient  than  the  likely  baseline 
or  alternative  decision.  In  fact,  it  is  difficult  to 
determine  whether  a  particular  CoSPA  application 
decreased  operational  effectiveness  of  strategic 
planning  without  the  benefit  of  post-evaluation 
data  analysis.  Limited  resources  precluded  a 


robust  analysis  of  potential  “negative"  applications 
of  CoSPA. 

Instances  where  inadequate  decisions  would 
be  made  based  on  CoSPA  were  not  unexpected. 
The  version  of  CoSPA  that  was  evaluated  in  2010 
was  not  the  full-capability  forecast  that  is 
envisioned.  Namely,  it  is  understood,  for  example, 
that  decisions  based  on  CoSPA  will  improve,  and 
delay  reduction  benefits  attributed  to  CoSPA  (see 


Section  4.3)  will  likely  increase,  when  the 
deterministic  forecast  is  complemented  by  a 
probabilistic  forecast  component  and  explicit 
measures  or  depictions  of  forecast  uncertainty. 
Additionally,  researchers  at  multiple  laboratories 
continue  to  research  and  develop  improvements  to 
the  extrapolation,  numerical  model,  and  blending 
components  of  the  CoSPA  forecast. 

4.2  Frequency  of  COSPA  Beneficial  Decisions 

The  frequency  of  CoSPA  benefits 
observations  at  each  facility,  normalized  by  the 
number  of  convective  weather  days  on  which 
observations  were  made  is  shown  in  Table  4-3. 
Several  interesting  (but  not  unexpected)  findings 
are  evident  with  these  results: 

•  Overall  CoSPA  usage  (including 
consultation  for  enhanced  situational 
awareness  and  decision  coordination)  was 
more  frequent  for  EAST  weather  impact 
events,  as  convection  in  the  Northeast 
NAS  quadrant  typically  causes  greater 
congestion  impacts  requiring  more 
intervention  and  strategic  planning 
initiatives. 

•  CoSPA  usage  was  most  frequent  at  "SCC” 
(Command  Center),  the  facility  charged 
with  overall  strategic  plan  development, 
execution,  and  management.  In  terms  of 
individual  benefits  categories,  ATCSCC 
most  frequently  used  CoSPA  in  support  of 
AFP,  Play  book  routes,  and  strategic 
rerouting  decisions.  Additionally,  ATCSCC 
was  much  more  active  in  using  CoSPA  for 
improved  situational  awareness,  for 
improved  impact  definition  within  CCFP 
polygons  (SP-CCFP),  and  for  enhanced 
inter/intra-facility  coordination  (which 
again,  is  not  surprising  given  that  the 
ATCSCC  often  facilitates  strategic 
planning  decisions  between  various  FAA 
facilities  and  airline  stakeholders).  In  fact, 
the  observed  daily  use  of  CoSPA  for 
situational  awareness  applications  was 
similar  to  the  observed  daily  use  of  other 
demonstration  tools  (i.e.,  CIWS  and 
RAPT)  in  their  second  or  third  year  of 
testing,  when  users  had  increased 
familiarity  and  “comfort"  with  the  new 
technology  [e.g.,  Robinson  et  al.  (2006); 
Robinson  et  al.  (2009)].  This  suggests  that 
more  applied  applications  of  CoSPA  for 
explicit  strategic  plan  development  may  be 
expected  in  any  follow-up  evaluations,  as 


users  translate  more  situational- 
awareness-type  applications  directly  into 
improved  weather  impact  mitigation 
decisions  [Robinson  et  al.  (2011)]. 

•  ZOB  was  the  most  active  EAST  ARTCC  in 
terms  of  using  CoSPA  to  aid  with  AFP, 
Playbooks,  and  strategic  reroute  decision¬ 
making  (AFP,  EPB,  and  ERP).  Again  this 
was  not  surprising,  since  ZOB  is  a  key  en 
route  center  for  strategic  planning  when 
adverse  weather  impacts  the  Northeast. 

•  The  airline  ATC  coordinators  and 
dispatchers  (particularly  COA  and  UAL) 
were  active  in  using  CoSPA  to 
strategically  plan  routes  around  weather 
for  individual  flights  or  city-pairs,  as  well  as 
make  decisions  about  the  need  for 
"company-only"  airport  Ground  Stops  or 
terminal  management  initiatives  (‘AFM’ 
benefit  category).  The  airlines  also  most 
frequently  used  CoSPA  to  monitor 
potential  terminal  impacts  (e.g.,  EWR, 
ORD,  ATL,  IAH)  in  the  2  -  8  hr  strategic 
time  period  ('SA-T  category).  Moreover, 
the  airline  AOCs  also  most  frequently 
used  CoSPA  to  better  prepare  departure 
operations  at  specific  airports  (primarily 
metro  NY)  -  proactively  asking  for  NY 
departure  “escape  routes”,  the  availability 
of  departure  offload  routings,  etc.  (‘DOP’ 
category). 

•  Overall  CoSPA  usage  at  ZNY  was 
minimal.  This  result  was  not  completely 
surprising  given  that  strategic  planning  for 
weather  impacts  in  the  Northeast  NAS 
quadrant  is  inherently  set  up  to  sufficiently 
manage  air  traffic  upstream  of  ZNY,  given 
that  (a)  ZNY's  small,  congested  airspace 
affords  little  opportunity  for  planned  impact 
mitigation  actions  (beyond  increased  route 
spacing)  and  (b)  the  greater  need  of  ZNY 
is  to  maximize  tactical  (0-2  hr) 
opportunities  for  weather  impact 
mitigation.  In  other  words,  with  weather  in 
the  Northeast  NAS  quadrant,  many 
strategic  actions  are  being  taken  by  other 
facilities  for  ZNY,  reducing  the  need  for 
ZNY  to  use  CoSPA  to  devise  additional 
strategic  plans. 

•  Across  all  facilities,  use  of  CoSPA  for 
enhanced  planning,  execution,  and 
management  of  Ground  Delay  Programs 
(GDP)  was  low.  This  may  have  been  due 
to  a  combination  of  factors  such  as  (a) 
timing  and  frequency  of  thunderstorm 
events  at  large  terminals  or  metroplexes 


(e.g.,  metro  NY,  metro  DC,  ORD,  ATL) 
and  (b)  lack  of  user  confidence  in  the  new 
forecast  for  such  specific  strategic 
planning  decisions.  The  ability  of  CoSPA 
to  accurately  and  consistently  resolve 
local  storm  impacts  2-8  hours  in  the 
future  on  the  spatial  scale  of  a  terminal  or 
TRACON  may  also  have  been  ill-suited 
without  some  additional  information  about 
forecast  uncertainty. 

The  frequency  of  observed  CoSPA  beneficial 
decisions  per  convective  weather  day  for  all  EAST 
and  MIDWEST  facilities  is  shown  in  Figure  4-5. 
Across  both  evaluation  regions,  AFP,  EPB,  and 


ERP  CoSPA  applications  were  the  most  prevalent 
"quantifiable”  benefits  categories  -  and  were 
therefore  considered  the  primary  decisions  upon 
which  CoSPA  delay  savings  estimates  were  based 
(see  Sections  4.3  and  4.4).  CoSPA  usage  at 
EAST  facilities  for  enhanced  Departure  Operations 
Planning  (DOP)  in  the  2-8  hr  time  period  was  also 
quite  frequent,  but  DOP  cases  were  not 
considered  for  delay  savings  calculations  since 
most  of  these  observed  uses  were  more 
qualitative  in  nature  (e.g.,  asking  for  availability  of 
departure  offload  routes  3-6  hours  before  they  will 
be  needed),  and  thus  delay  reduction  benefits 
were  more  difficult  to  estimate. 


Table  4-3 

Normalized  CoSPA  Benefits  Observations  by  Facility**  for  (A)  EAST  and  (B)  MIDWEST  Evaluations 
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Figure  4-5.  Frequency  of  observed  CoSPA  benefits  per  convective  weather  day  for  all  EAST  and  MIDWEST  facilities. 


4.3  COSPA  Delay  Savings  Case  Studies 

Case  studies  of  observed  CoSPA  operational 
usage  were  analyzed  to  estimate  the  delay 
savings  associated  with  the  three  primary  strategic 
planning  benefits  categories  -  AFP,  EPB,  and 
ERP.  To  prevent  double-counting,  care  was  taken 
when  categorizing  CoSPA  usage  observations  to 
ensure  that  one  primary  benefit  category  was 
assigned  to  each  documented  application  (e.g.,  if 
a  strategic  rerouting  application  was  categorized 
as  EPB,  then  it  could  not  also  be  categorized  as 
ERP  -  and  vice  versa). 


A  total  of  15  CoSPA  benefits  cases  were 
analyzed  -  5  cases  for  each  primary  benefits 
category.  Cases  were  arbitrarily  selected  in  a 
manner  that  allowed  us  to  sample  beneficial 
decisions  and  examine  potential  delay  reduction 
benefits  across  as  many  different  facilities  and 
airspace  regions  as  possible.  A  summary  of  case 
study  results  from  one  AFP,  EPB,  and  ERP 
decision  derived  from  CoSPA  is  presented  in 
Tables  4-4  -  4-6,  respectively. 


Table  4-4.  Summary  of  CoSPA  Benefits  Case  Study  for  Improved  AFP  Decision-Making 


Date: 

19  July  2010 

Facilities  Using  CoSPA: 

ZDC 

CoSPA  Application: 

Upon  reaching  decision  to  implement  AFP  A08,  ZDC  used  CoSPA  3  - 
5  hr  precipitation  forecasts  -  predicting  minimal  Northeast  convective 
weather  development  prior  to  1900  UTC  -  to  convince  ATCSCC  to 
start  AFP  at  1900  UTC  rather  than  1700  UTC. 

Alternative  Decision: 

Without  CoSPA,  AFP  A08  likely  would  have  started  at  1700  UTC;  1300 
UTC  4  hr  CCFP  predicted  low  confidence,  low  coverage  polygon 
through  central  ZDC  at  1700  UTC  -  historically,  this  type  of  CCFP 
forecast  has  proven  sufficient  justification  for  A08  implementation. 

Benefit  Period: 

1700-1900  UTC  (2.0  hr) 

CoSPA  Delay  Savings  Calcu 

lations: 

A08  Program:  1900  -  0200  UTC  (issued  1445  UTC) 

Rate:  90  flights/hr 

Avg  Delay:  16  min 

Total  flights  departing  for  primary  A08  airports  between  1700-1900  UTC  AND  crossing  A08  boundary  that 

would  have  received  AFP  EDCTs,  averaging  16  min  delay: 

141 

Primary  delay  reduction  for  141  departures  avoiding  16  min  AFP  delay: 

37.6  hr 

Downstream  Delay  Reduction: 

21.4  hr 

Total  CoSPA  Delay  Reduction: 

59.0  hr 

Direct  Operating  Cost  (DOC)  Savings: 

$36,359 

Passenger  Value  Time  (PVT)  Savings: 

$47,301 

Downstream  Cost  Savings  (PVT  only): 

$26,921 

Total  CoSPA  Cost  Savings: 

$110,581 

Table  4-5.  Summary  of  CoSPA  Benefits  Case  Study  for  Enhanced  Playbook  Route  Management 
_ , _ (EPB) _ 


Date: 

14  June  2010 

Facilities  Using  CoSPA: 

ZID 

CoSPA  Application: 

Z1D  used  CoSPA  precipitation  forecast  (as  additional  information  to 
augment  CCFP)  to  determine  that  an  eastward  moving  cluster  of 
storms  was  predicted  to  hold  together  and  impact  ORD  arrivals 
through  ZID  airspace.  Based  on  CoSPA,  ZID  proactively  collaborates 
on  a  Playbook  reroute  plan  -  “ORD-East-Arrivals"  modified  Playbook 
route  implemented  2055-0100  UTC  -  allowing  a  single  stream  of  ORD 
arrival  traffic  through  ZID  airspace 

Alternative  Decision: 

Without  CoSPA,  ZID  likely  would  not  have  readily  anticipated  the 
functional  gaps  in  forecast  weather  within  the  large,  high-confidence 
CCFP  forecast  region.  In  turn,  single  stream  ORD  arrival  flows  over 
MZZ/FWA  (part  of  modified  Playbook  routing)  likely  would  have 
instead  routed  further  west  to  the  FAM  fix  or  further  north  via  FWA 
(rather  than  using  MZZ) 

Benefit  Period: 

2055-2347  UTC  (2.9  hr) 

CoSPA  Delay  Savings  Calcu 

lations: 

Number  of  ORD  arrival  flights,  departing  during  benefits  period,  utilizing  MZZ  or  FWA  leg  of  Playbook 
routing:  12 

Total  flight  distance  saved:  1450  mi 

Primary  delay  reduction: 

4.0  hr 

Downstream  Delay  Reduction: 

2.3  hr 

Total  CoSPA  Delay  Reduction: 

6.3  hr 

Direct  Operating  Cost  (DOC)  Savings: 

$17,156 

Passenger  Value  Time  (PVT)  Savings: 

$  5,032 

Downstream  Cost  Savings  (PVT  only): 

$  2,893 

Total  CoSPA  Cost  Savings: 

$25,081 

Table  4-6.  Summary  of  CoSPA  Benefits  Case  Study  for  Enhanced  Strategic  Management  of  Air 
_ Traffic  Reroutes  (ERP) _ 


Date: 

07  July  2010 

Facilities  Using  CoSPA: 

ZMP 

CoSPA  Application: 

At  approximately  1500  UTC,  ZMP  area  supervisor  wanted  to  stop  the 
NW  arrival  flow  to  ORD  through  the  southeast  ZMP  sectors  because  of 
convection  along  the  ZAU/ZMP  border.  The  STMC  viewed  CoSPA 
forecasts  -  which  predicted  storms  continuing  on  this  region  but 
weakening  below  30  kft  with  time  (+3  hr  and  beyond)  -  and  convinced 
the  area  supervisor  to  keep  the  nominal  route  open. 

Alternative  Decision: 

Without  CoSPA,  the  GEP  (northwest)  ORD  arrival  flow  would  have 
been  rerouted  south,  to  the  westerly  arrival  flow,  per  the  area 
supervisor's  request 

Benefit  Period: 

1515-1815  UTC  (3.0  hr) 

CoSPA  Delay  Savings  Calcu 

ations: 

Number  of  ORD  flights  flying  nominal  ZMP-GEP  (northwest)  arrival  route  and  avoiding  reroute  to  the 
south:  8 

Total  flight  distance  saved:  784  mi 

Primary  delay  reduction: 

2.1  hr 

Downstream  Delay  Reduction: 

1.2  hr 

Total  CoSPA  Delay  Reduction: 

3.3  hr 

Direct  Operating  Cost  (DOC)  Savings: 

$  9,007 

Passenger  Value  Time  (PVT)  Savings: 

$  2,642 

Downstream  Cost  Savings  (PVT  only): 

$  1,510 

Total  CoSPA  Cost  Savings: 

$13,159 

A  summary  of  CoSPA  delay  savings  (hours  of 
delay  saved  and  cost  savings)  derived  from  all  of 
the  cases  studies,  for  each  of  the  three  primary 
benefits  categories,  is  provided  in  Tables  4-7A  - 
4-7C.  The  calculated  delay  reduction  for  individual 
beneficial  strategic  decisions  derived  from  CoSPA 
ranged  from  3.3  hours  (07  July  2010  -  ERP  case) 
to  145.5  hours  (20  July  2010  -  AFP  A08  case). 
Case-to-case  variability  in  delay  savings  followed 
expected  trends,  with  the  largest  benefits  being 
associated  with  AFP  improvements  (AFPs  are 
longer,  more  encompassing  strategic  initiatives 
that  can  affect  many  more  aircraft)  and  the 
smallest  benefits  being  associated  with  enhanced 
reroute  planning  (ERP),  typically  affecting  the 
fewest  aircraft  in  the  2  -  8  hr  decision  period. 
Moreover,  benefits  associated  with  Playbook 
reroute  enhancements  (EPB)  were  anticipated  to 
be  greater  than  ERP  benefits,  since  the  former 
typically  affect  more  aircraft  than  the  latter. 

The  results  from  one  case  study  -  14  June 
2010  ERP  case  -  yielded  no  strategic  ATM  delay 
savings  (see  Table  4-7C).  Inspection  of  this  case 
reveals  that  though  the  2  -  3  hour  (i.e.,  strategic) 
CoSPA  forecasts  were  used  to  identify  a  reroute  of 
ATL  arrival  traffic  from  the  Northeast  through  ZID 


airspace,  only  three  aircraft  that  were  already 
airborne  and  within  two  hours  of  arriving  at  ATL 
(even  when  using  the  new  route)  utilized  the  route. 
Since  the  CoSPA  forecast  was  used  to  manage 
only  a  tactical  decision  -  and  no  other  airborne 
flights  or  additional  flights  waiting  to  depart  used 
the  CoSPA-identified  ZID  reroute  -  the  strategic 
benefits  for  this  decision  were  estimated  to  be 
zero.  Since  this  case  was  arbitrarily  selected,  and 
the  observed  CoSPA  application  was  categorized 
as  'ERP’,  the  results  were  included  with  the  other 
ERP  case  study  results,  to  represent  other 
potential  CoSPA  benefits  cases  that  may,  upon 
closer  inspection,  yield  little  or  no  strategic 
planning  benefits. 

Median  CoSPA  delay  savings  per  primary, 
quantified  benefits  category  were  computed  to 
determine  the  average  delay  reduction  per  CoSPA 
application  (Table  4-8).  Mean  delay  estimates  are 
likely  the  most  appropriate  measure  for  average 
delay  but  given  these  small  case  study  sample 
sets,  and  the  degree  of  spread  between  some 
benefits  estimates,  high  variability  exists  with  the 
mean.  Therefore,  median  benefits  results  were 
preferred  for  estimating  annual  CoSPA  savings. 


Table  4-7  A 

CoSPA  Benefit  1:  Improved  Development,  Execution,  and  Management  of  Airspace  Flow 

Programs  (AFP) 


DELAY  SAVED  (hr) 

SAVINGS  ($) 

Date 

AFP 

Primary 

Downstream 

Total 

Direct 

Operating 

Costs 

(DOC) 

Passenger 

Value 

Time 

(PVT) 

Passenger 
Value  Time 
Downstream 
(PVTd) 

Total 

16  Jun 

A05 

36.5 

20.8 

57.3 

40,943 

45,917 

26,166 

113,026 

19  Jul 

A08 

37.6 

21.4 

59.0 

36,359 

47,301 

26,921 

110,581 

20  Jul 

A08 

92.7 

52.8 

145.5 

89,641 

116,617 

66,422 

272,680 

21  Jul 

A05 

24.7 

14.1 

38.8 

23,885 

31,073 

17,738 

72,696 

05  Aug 

A05 

23.6 

13.4 

37.0 

25,607 

29,689 

16,857 

72,153 

Table  4-7B 

CoSPA  Benefit  2:  Enhanced  Playbook  Reroute  Planning  and  Execution  (EPB) 


DELAY  SAVED  (hr) 

SAVINGS  ($) 

Date 

Facility 

Primary 

Downstream 

Total 

Direct 

Operating 

Costs 

(DOC) 

Passenger 

Value 

Time 

(PVT) 

Passenger 
Value  Time 
Downstream 
(PVTd) 

Total 

14  Jun 

ZID 

4.0 

2.3 

6.3 

17,156 

5,032 

2,893 

25,081 

06  Jul 

ZAU 

1.3 

0.7 

2.0 

5,576 

1,635 

881 

8,092 

21  Jul 

ZBW 

3.4 

1.9 

5.3 

8,604 

4,277 

2,390 

15,271 

20  Jul 

see 

25.5 

14.5 

40.0 

24,658 

32,079 

18,241 

74,978 

04  Aug 

ZOB 

24.0 

13.7 

37.7 

23,208 

30,192 

17,235 

70,635 

Table  4-7C 

CoSPA  Benefit  3:  Enhanced  Reroute  Planning  -  Strategic  Planning  Period  (ERP) 


DELAY  SAVED  (hr) 

SAVINGS  ($) 

Date 

Facility 

Primary 

Downstream 

Total 

Direct 

Operating 

Costs 

(DOC) 

Passenger 

Value 

Time 

(PVT) 

Passenger 
Value  Time 
Downstream 
(PVTd) 

Total 

21  Jul 

COA 

4.5 

2.6 

7.1 

4,352 

5,661 

3,271 

13,284 

04  Aug 

ZOB 

1.6 

0.9 

2.5 

6,862 

2,013 

1,132 

10,007 

06  Jul 

ZAU 

4.4 

2.5 

6.9 

9,038 

5,535 

3,145 

17,718 

07  Jul 

ZMP 

2.1 

1.2 

3.3 

9,007 

2,642 

1,510 

13,159 

14  Jun 

ZID 

0 

0 

0 

0 

0 

0 

0 

Table  4-8 

Median  CoSPA  Delay  Savings  per  Benefits 
_ Category _ _ 


CoSPA  Benefit 
Category 

Delay  Saved 
(hr) 

Cost 

Savings 

($) 

Median  AFP 

57.3 

113,026 

Median  EPB 

6.3 

25,081 

Median  ERP 

3.3 

13,159 

4.4  ESTIMATED  ANNUAL  COSPA  DELAY 
SAVINGS 

The  normalized  frequency  of  observed  CoSPA 
benefits  presented  in  Table  4-3  (with  the  exception 
of  ‘AFP’  applications)  were  summed  up  across  all 
operational  facilities  and  averaged  between  EAST 
and  MIDWEST  facilities  to  determine  the  total 
observed  benefits  frequency  per  convective 
weather  day.  Averaging  the  EAST  and  MIDWEST 
total  frequency  of  occurrence  of  CoSPA  benefits 
ensures  that  results: 

(a)  Coincide  with  the  methodology  of 
estimating  annual  benefits  by  considering 
the  collective  evaluation  airspace  (9 
ARTCCs)  as  one  arena  for  strategic 
planning  benefits  -  with  weather  impact 
mitigation  opportunities  estimated  to  be 
present  on  95  annual  days  (see  Section 
3.4); 


(b)  Avoid  "double-counting”  the  potential 
benefits  frequency  (e.g.,  ATCSCC  has 
normalized  benefits  frequency  tallies  for 
EAST  and  MIDWEST  evaluations  -  these 
“per  day”  statistics  need  to  be  averaged 
together) 

AFP  benefits  were  handled  differently.  Since, 
within  the  evaluation  region  of  this  study,  AFPs 
required  for  weather  impact  mitigation  have 
historically  only  been  implemented  when 
convection  is  present  or  expected  in  an  EAST 
facility,  it  did  not  make  sense  to  average  CoSPA 
AFP  benefits  frequencies  between  the  EAST  (2.1 
per  day)  and  the  MIDWEST  (0  per  day)  facilities. 
The  total  AFP  benefits  frequency  from  EAST 
evaluations  was  used  as  the  total  benefits 
frequency. 

Multiplying  the  estimated  CoSPA  benefits 
frequency  (and  AFP  benefits  frequency)  per 
convective  weather  day  by  an  estimated  95 
weather  impact  days  (55  ‘AFP-consideration’ 
days)  yields  the  annual  CoSPA  benefits  frequency 
(Table  4-9). 

Annual  CoSPA  delay  reduction  benefits, 
computed  from  median  delay  savings  per  CoSPA 
use  and  the  annual  CoSPA  frequency  of  use,  are 
presented  in  Table  4-10.  Combined  for  the  three 
primary  benefits  categories,  annual  CoSPA 
benefits  estimates,  derived  from  real-time 
observations  across  all  FAA  and  airline  facilities 
included  in  the  2010  evaluation  were  10,000 
hours  of  delay  saved,  with  a  cost  savings  of 
$26.8  M. 


Table  4-9 


Annual  CoSPA  Benefits  Frequency  per  Category  * 


I 

|l. 

AFP 

115] 

2. 

GDP 

28 

3. 

GS 

9 

4. 

EPB 

332 

5. 

ERP 

418! 

6.  MRR 

57 

7.  MIL 

9 

8.  ITR 

19 

9.  RIF 

76 

10.  AFM 

66 

11.  DOP 

152 

12.  RCM 

19 

13.TMA 

38 

14.  ICR 

9 

15.SPD 

646 

16.  SP-CCFP 

513 

17.  I/IC 

2113 

18.  SA-R 

3771 

19.  SA-T 

1895 

20.  SA 

3125 

21.  STAFF 

47 

22.  EQUIP 

9 

23.  CWSU 

313 

*  Benefit  categories  in  red  boxes  used  to 
estimate  annual  delay  savings 


Table  4-10 

Annual  CoSPA  Delay  Reduction  Benefits 


Hours 

Monetary  Value  ($) 

CoSPA  Benefit 
Category 

Primary 

Downstream 

TOTAL 

DOC 

PVT 

PVT 

downstream 

TOTAL 

AFP 

4,197 

2,392 

6,589 

4,708,445 

5,280,455 

3,009,090 

12,997,990 

EPB 

1,328 

763 

2,091 

5,695,792 

1,670,624 

960,476 

8,326,892 

ERP 

878 

502 

1,380 

3,764,926 

1,104,356 

631,180 

5,500,462 

TOTAL 

6,403 

3,657 

10,060 

14,169,163 

8,055,435 

4,600,746 

26,825,344 

5.  CONCLUSIONS 

Effective  air  traffic  management  (ATM) 
strategic  planning  is  critical  for  mitigating  delays 
and  ensuring  a  safe  and  efficient  operation  during 
NAS  weather  impact  events.  An  accurate  and 
operationally  useful  strategic  weather  forecast  (2  - 
8  hour  forecast)  is  key  decision  support 
information  used  by  FAA  traffic  managers  and 
airline  air  traffic  control  (ATC)  coordinators  and 
dispatchers  to  proactively  anticipate  and  plan  for 
the  onset  and  cessation  of  operationally  significant 
weather  constraints.  A  recent  advance  in  strategic 
weather  forecast  technology  is  the  CoSPA 
forecast.  CoSPA  is  a  high  resolution  (3  km)  0-8 
hr  deterministic  "radar-like"  forecast  (updating 
every  15  minutes)  of  precipitation  and  echo  tops. 
CoSPA  uses  CIWS  short-term  forecast 
techniques,  numerical  weather  predictions  from 
the  High  Resolution  Rapid  Refresh  (HRRR)  model 
for  longer-term  forecasts,  and  forecast  “blending” 
technologies  to  produce  seamless  predictions  of 
how  convection  may  evolve  and  move  with  time. 
The  CoSPA  product  was  fully  integrated  with 
CIWS  and  was  available  on  the  CIWS  platform  at 
facilities  with  dedicated  situation  displays. 

In  2010,  a  field  evaluation  was  conducted  at 
17  FAA  and  airline  dispatch  facilities  to  assess  the 
operational  performance,  strategic  planning  utility, 
and  estimated  delay  reduction  benefits  of  the 
CoSPA  weather  forecast.  Simultaneous,  real-time 
observations  of  CoSPA  usage  were  made  on  15 
convective  weather  days  (181  hours  of 
observations)  during  June  -  September  2010. 

An  assessment  of  the  CoSPA  operational 
performance  during  the  real-time  evaluations 
revealed  that  the  overall  enhanced  capabilities  of 
CoSPA  (compared  with  the  baseline  CCFP)  to 
predict,  with  high  spatial  and  temporal  resolution, 
convective  weather  initiation,  decay,  organization, 


coverage,  intensity  (including  vertical  extent),  and 
storm  motion  were  valuable  weather  forecast 
attributes  for  aviation  planners  making  strategic 
ATM  decisions.  Often,  CoSPA  forecasts  were 
used  to  better  define  the  specific  regions,  timing, 
and  severity  of  potential  convective  weather 
constraints  within  the  more  broad  and  general 
CCFP  forecast  polygons,  leading  to  enhanced 
situational  awareness,  improved  strategic  planning 
coordination,  and  ultimately,  more  effective  traffic 
management  decisions  and  ATM  initiatives. 

CoSPA  forecasts  were  observed  on  occasion 
to  over-predict,  under-predict,  and/or 
inconsistently  predict  convective  weather 
coverage  and  severity.  At  times,  these  forecasts 
contributed  to  reduced  common  situational 
awareness  and  less  efficient  assessments  of 
anticipated  weather  constraints  during  the 
strategic  planning  period  (2-8  hours).  As  a 
prototype  forecast  technology  still  in  development, 
research  continues  to  improve  the  forecast 
capabilities  of  CoSPA.  A  targeted  improvement  - 
identified  both  by  developers  and  the  operational 
users  -  is  to  include  some  type  of  forecast 
uncertainty  measure  or  metric  (perhaps  through 
the  use  of  probabilistic  forecasts)  that  would 
inform  users  of  situations  when  the  accuracy  or 
consistency  of  the  deterministic  weather 
predictions  do  not,  at  that  time,  support  specific 
strategic  planning  actions. 

Twenty-three  unique  CoSPA  benefits 
categories  were  identified  during  the  operational 
field-use  evaluation.  Observed  CoSPA 
applications  included  quantifiable  benefits  through 
improved  strategic  planning  initiatives  and 
enhanced  situational  awareness  and  improved 
collaborative  decision-making.  Specifically, 
CoSPA  was  used  to  assist  with  Airspace  Flow 
Program  (AFP)  decisions  about  program  traffic 


rates,  timing  for  implementation,  when  to  cancel 
an  AFP,  or  even  if  an  AFP  was  needed.  CoSPA 
was  also  used  to  assist  with  Playbook  reroute 
execution  and  management.  Smaller,  more 
surgical  strategic  reroute  decisions  were  also 
planned  based  upon  CoSPA  forecasts  of  impacted 
and  viable  airspace.  Aviation  planners  at  Airline 
Operations  Centers  (and  some  FAA  facilities) 
were  observed  using  CoSPA  to  make  strategic 
routing  decisions  for  individual  flights  or  city-pairs. 
Finally,  CWSU  meteorologists  in  FAA  En  Route 
Centers  and  weather  specialists  at  the  ATCSCC 
were  also  observed  using  CoSPA  to  assist  with 
their  tasks  -  which  included  CCFP  collaboration 
and  development. 

CoSPA  benefits  case  studies  were  analyzed  in 
an  effort  to  quantify  the  estimated  delay  savings 
associated  with  the  three  primary  CoSPA  strategic 
weather  impact  mitigation  benefit  categories 
[improved  AFP  execution  and  management  (AFP), 
enhanced  Playbook  reroute  planning  and 
execution  (EPB),  and  enhanced  strategic  reroute 
planning  (ERP)].  Results  show  that  per-use 
CoSPA  benefits  ranged  from  3.3  hours  to  145.5 
hours.  The  variability  in  case-to-case  delay 
savings  followed  expected  trends,  with  the  largest 
benefits  being  associated  with  AFP  improvements 
and  the  smallest  benefits  being  associated  with 
enhanced  reroute  planning  (ERP),  which  typically 
affected  the  fewest  aircraft  in  the  2  -  8  hr  decision 
period.  Relatively  frequent  usage  of  CoSPA  for 
Departures  Operations  Planning  assistance  (DOP) 
was  also  observed,  but  not  quantified,  since  most 
of  these  observed  uses  were  more  qualitative  in 
nature,  making  it  difficult  to  estimate  delay  savings 
for  this  benefits  category. 

The  frequency  of  each  type  of  CoSPA 
application  was  tabulated  for  each  FAA  and  airline 
facility  and  rolled-up  to  an  annual  CoSPA  benefits 
frequency  estimate  based  upon  historical 
averages  of  weather  impact  events  (accounting  for 
longer-term  climatology).  Median  case  study  delay 
savings  per  AFP,  EPB,  and  ERP  benefit  category 
were  multiplied  by  the  estimated  annual  frequency 
of  these  CoSPA  beneficial  decisions  to  estimate 
the  annual  CoSPA  delay  reduction  benefits  for 
2010.  Estimated  annual  CoSPA  benefits  in  2010 
totaled  10,000  hours  of  delay  saved,  with  a  cost 
savings  of  $26.8  M. 

In  addition  to  the  continued  research  into 
forecast  improvements,  CoSPA  benefits  are  also 
expected  to  increase  as  ATM  strategic  planning 
tools  continue  to  evolve  and  new  capabilities  and 
procedures  become  available  that  may  take  better 
advantage  of  a  high-resolution,  rapidly-updating 
deterministic  precipitation  and  echo  tops  forecast. 


Moreover,  significant  improvements  to  strategic 
planning  and  weather  impact  mitigation  are 
anticipated  once  CoSPA  weather  forecasts  are 
translated  into  ATM  impact  predictions  and 
integrated  into  current  and  planned  ATM  decision 
support  capabilities  (Lin  et  al.  2011). 
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